Sensorimotor assessment and training

ABSTRACT

Disclosed herein are system, method, and computer program product embodiments for assessing performance of a player of a game. An embodiment operates by monitoring for an input from the player of the game, receiving the input from the player of the game, and determining a characteristic of the game resulting from the input from the player. Based on the input from the player, a performance of the player is assessed. The performance of the player relating to one or more metrics of the game is monitored, and is assessed by comparing the input from the player during the period of time to an optimal input during the period of time in the game.

PRIOR APPLICATION DATA

The present application is a continuation of prior U.S. application Ser.No. 18/083,928, filed on Dec. 19, 2022, entitled “SENSORIMOTORASSESSMENT AND TRAINING”, which is a continuation of prior U.S.application Ser. No. 17/975,769, filed on Oct. 28, 2022, entitled“SENSORIMOTOR ASSESSMENT AND TRAINING”, which is a continuation of priorU.S. application Ser. No. 16/121,210, filed on Sep. 4, 2018, entitled“SENSORIMOTOR ASSESSMENT AND TRAINING”, which in turn claims the benefitof prior U.S. provisional application 62/554,212, each incorporated byreference herein in its entirety.

TECHNICAL FIELD

This disclosure relates generally to assessing and training a player ofa video game.

BACKGROUND

Competitive video gaming (eSports) is the fastest growing sport on theplanet with an estimated 380 million players globally. The global videogame market was $138B in 2018 with 13% year-over-year growth.Professional eSports was $1B in 2017 with 38% year-over-year growth.

eSports players want to win; losing is painful. In traditional “stickand ball” sports, a player can improve their chances of winning by 1)assessing their own abilities and training to improve their performance,and 2) assessing the performance of other players to choose betterteammates and to choose strategies that expose their opponents'weaknesses. Unlike traditional sports, there are few metrics in eSportsfor assessing a player's ability and health, and no training methods forimproving a player's performance. Amateur eSports players andindividuals in the professional eSports industry agree that assessmentand training are critical needs. Burnout is high among many players dueto inefficient training. Top players are well aware that they haveweaknesses in their game and they spend a lot of time trying to identifythem and correct them.

SUMMARY

According to an embodiment, a computer-implemented method for assessingperformance of a player of a game is provided. The computer-implementedmethod comprises: (i) monitoring, by a processor, for an input from theplayer of the game over a period of time; and (ii) receiving, by theprocessor, the input from the player of the game during the period oftime; (iii) determining, by the processor, a characteristic of the gameresulting from the input from the player; and (iv) assessing, by theprocessor, based on the input from the player during the game, aperformance of the player during a period of time in the game, theperformance of the player relating to one or more metrics of the game.The performance of the player comprises comparing the input from theplayer during the period of time in the game to an optimal input fromthe player during the period of time in the game.

According to another embodiment, a system comprising a memory andprocessor coupled to the memory is provided. The processor is configuredto: (i) monitor an input of a player of a game during a period of time,(ii) monitor the input from the player during the game during the periodof time, (iii) determine a characteristic of the game from the inputfrom the player, and (iv) assess, based on the input from the playerduring the game, a performance of the player during a period of time inthe game. The performance of the player relates to one or more metricsof the game, and it comprises comparing the input from the player duringthe period of time in the game to an optimal input from the playerduring the period of time in the game.

In some embodiments, a system comprises a computer having a computerprogram that implements an eSport game that assesses a player's ability(e.g., speed and accuracy), and that trains the player to improve theirperformance. The computer program presents sensory stimulation (e.g.,images on a computer display) that corresponds to a 3D virtualenvironment. The computer program also receives input signals from aninput controller (e.g., mouse and keyboard) to measure the player'smovements. The computer program changes the state of the virtualenvironment based on the input signals, and re-renders the sensorystimulation dynamically over time according to the changes of state ofthe virtual environment. The computer program, furthermore, evaluatesthe input signals from the input controller, to assess the player'sperformance. Finally, the computer program dynamically changes thesensory stimulation and/or the evaluation of the input signals to trainthe player to improve their performance.

According to yet another embodiment, a tangible computer-readable devicehaving instructions stored thereon is provided. The tangiblecomputer-readable device, when executed by a computing device, causesthe computing device to perform operations comprising: (i) monitoringfor an input from a player of a game over a period of time; (ii)receiving the input from the player during the game over the period oftime; (iii) determining a characteristic of the game resulting from theinput from the player; and (iv) assessing, based on the input from theplayer during the game, a performance of the player. The performance ofthe player is related to one or more metrics of the game, and itcomprises comparing the input from the player during the period of timein the game to an optimal input from the player during the period oftime in the game.

In some embodiments, a tangible computer-readable device is providedhaving instructions stored thereon that, when executed by a computingdevice, permits the computing software platform to provide afirst-person shooter eSport game (e.g., “Overwatch”), and to performoperations to assess a player's ability (e.g., speed and accuracy) andtrain the player to improve their performance.

According to yet another embodiment, a smart input controller isprovided that identifies systematic errors in a player's movements andautomatically compensates for the errors to improve the player'sperformance.

A computer-implemented method may include that the determining of therelationship and the manipulation of the outcome of the input of theplayer are performed by a computer program comprising the game. Acomputer-implemented method may include that the determining of therelationship and the manipulation of the outcome of the input of theplayer is performed by a device for playing the game.

The manipulation of the outcome of the input of the player may includemanipulating at least one of a position, velocity, acceleration,higher-order temporal derivative of the position, orientation, angularvelocity, angular acceleration, higher-order temporal derivative of theorientation, joint angle, angular velocity of the joint angle, angularacceleration of the joint angle, and higher-order temporal derivativesof the joint angle of the input of the player.

BRIEF DESCRIPTION OF THE FIGURES

The accompanying drawings are incorporated herein and form a part of thespecification.

FIG. 1 illustrates a screenshot from a game providing assessment andtraining of a player, according to some embodiments.

FIG. 2 illustrates exemplary trajectories of movement of a player in thegame of FIG. 1 , according to some embodiments.

FIG. 3 illustrates exemplary trajectories of movement of a player over aperiod of time in the game of FIG. 1 , according to some embodiments.

FIG. 4 illustrates an exemplary variability of a player's movement timecourses during a previously played game described in FIG. 1 , accordingto some embodiments.

FIG. 5 illustrates a process for assessing one or more metrics of aplayer of a game, according to some embodiments.

FIG. 6 illustrates a screen shot of a scorecard presented to a playerafter a round previously played in the game of FIG. 1 , according tosome embodiments.

FIG. 7 illustrates a screen shot of a scorecard that summarizes aplayer's performance for multiple rounds played in the game of FIG. 1 ,according to some embodiments.

FIG. 8 illustrates a screen shot comparing a player's performance withothers who have played while the player plays the game of FIG. 1 ,according to some embodiments.

FIG. 9 illustrates a screenshot displaying a player's improvement overtime from the game described in FIG. 1 , according to some embodiments.

FIG. 10 illustrates an exemplary screenshot from a game that providesassessment and training capabilities of a player of a game, according tosome embodiments.

FIG. 11 illustrates an exemplary speed-accuracy tradeoff curves of aplayer in a game, according to some embodiments.

FIG. 12 illustrates a process for detecting a cheater in a game,according to some embodiments.

FIG. 13 illustrates exemplary systematic biases of a person's movementsin a game, according to some embodiments.

FIG. 14 illustrates a process for assessing performance of a player of agame, according to some embodiments.

FIG. 15 illustrates an example computer system useful for implementingvarious embodiments.

In the drawings, like reference numbers generally indicate identical orsimilar elements. Additionally, generally, the left-most digit(s) of areference number identifies the drawing in which the reference numberfirst appears.

DETAILED DESCRIPTION

Provided herein are system, method and/or computer program productembodiments, and/or combinations and sub-combinations thereof, forassessing a player's ability in an environment based on their input, andfor training the player to improve (and even optimize) their performancein the environment such that they acquire an improved or even optimalinput. As used herein, in embodiments, optimal input is input thatresults in a more or most favorable outcome or performance. For example,an optimal input can be that of an expert. Moreover, some embodimentsare directed to a player's ability and/or performance in a gamingenvironment. The factors that can be assessed include those related aplayer's sensorimotor behavior, such as perception, attention, motorcontrol, perceptual learning, motor learning, perceptually-guideddecision-making, and cognitive control of sensorimotor behavior.However, a person of ordinary skill would readily recognize that thismay apply a wide-variety of other applications which depend on sensorybehavior, such as rehabilitation, military training, and policetraining, to name just some examples. The same factors being discussedand assessed in the present application also apply to theseapplications.

FIGS. 1 to 9 illustrate exemplary games which provide assessment andtraining capabilities of a player's ability and/or performance. Thegames can be practice games, or can be real-games. To assess the abilityof a player, a number of metrics are monitored. The metrics can dependon the game. The metrics can also be selected by a player and/or atrainer of the game. Exemplary metrics include speed, precision,accuracy, reaction time. The metrics can also relate to one or morecharacteristics of a player in a game, such as shooting of a weapon,interaction with a ball (e.g., baseball, soccer ball, tennis ball),virtual combat with another player.

The game can include several different scenarios for target practice. Ina first scenario, the “Spidershot Accuracy” scenario, a player can shootat one or more targets presented at random locations in a virtualenvironment. Targets can be visible for a fixed period of time, and theplayer's goal can be to accurately eliminate all targets that appear.This scenario measures accuracy orienting to targets at differentlocations. A second scenario, the “Spidershot Speed” scenario, issimilar to the Spidershot Accuracy scenario but the target presentationdurations are varied to measure the speed with which the player orientsto different locations. Targets are presented, one at a time, at variouslocations in a 3D virtual environment. Each target is presented for alimited period of time. The player's goal is to shoot each target beforeit disappears. The presentation time of each target is dynamicallyadjusted based on the player's performance at that location, to trainthe player to improve their performance. When the player hits a target,the next target at that same location is presented for a shorter periodof time. When a target is missed, the next target at that same locationis presented for a longer period of time. In a third scenario, the“Reflexshot” scenario, one or more targets are presented at randomlocations and at random intervals. The player's goal is to quickly andaccurately eliminate as many targets as possible before a time limit isreached. In a fourth scenario, the “Headshot” scenario, targets areagain presented at random locations. The player receives points foraccurate headshots, and loses points proportional to the distance froman accurate headshot (e.g., the greater the distance, the greater theloss of points). In a fifth scenario, the “Cornershot/Pentakill”scenario, the player is positioned near the end of a wall. A pluralityof enemies appear through an entrance in the wall and travel across thevirtual environment to a destination. The player must eliminate allenemies before they reach their destination. This scenario measuresaccuracy for multiple moving targets, and for both left corner and rightcorner positions. This is a common strategic position in first personshooter (FPS) games. A player will camp the exit of an area to quicklyeliminate enemies as they pass from the exit into the next cover area.Speed and accuracy is critical because the player has the element ofsurprise initially. After the first shot, the enemy knows where theyare. In a sixth scenario, the “Strafeshot” scenario, a single targetstrafes unpredictably. The goal of the player is to hit the movingtarget as many times as possible in a limited amount of time. Additionalmodes include movement techniques unique to particular characters inpopular eSports (e.g., Genji's double jump, Tracer's blinkteleportation, and Pharah's rocket boost/hover combo). Strafeshotaccuracy is critical because: (1) rarely are players shooting at stillenemies and (2) upon engagement, most enemies strafe as a defensivetactic. In a seventh scenario, the “Freeplay” scenario, a player isdropped into a typical FPS map with artificial intelligence (AI) enemiesthat shoot back. The player's goal is to survive as long as possible andeliminate as many enemies as possible. This scenario measures a numberof skills and biases that impact the player's performance, includingaccuracy while moving, accuracy while being attacked, movement andpositioning biases, and general gameplay biases.

FIG. 1 shows a screenshot of an exemplary scenario of the game. In thisscenario, targets are presented at each of 8 locations. For each targetlocation, targets are presented for 3 different durations and 2different sizes. For each target location, duration, and size, there area plurality of repeated trials. The different target locations,durations, and sizes are presented randomly and shuffled in-order.Movements are sampled at 60 Hz in this example, although other samplingrates could be used.

FIG. 2 shows exemplary trajectories of a person's movement while theywere playing the game of FIG. 1 . The movement trajectories weredetermined by sampling an input position from an input controller,controlled by a person while the person plays the game. The inputposition can control a number of features of the player, includingmovement of the player and/or one or more actions of a player (e.g.,utilization of a weapon). The numbers in FIG. 2 indicate the differenttargets (represented by asterisks). The small dots indicate the medianof the player's movement trajectories, across several repeated trialsfor each target location. The circles represent the best-fits of a modelof each movement trajectory.

FIG. 3 shows movement time courses corresponding to the movementtrajectories of FIG. 2 . The top panel of FIG. 3 plots the horizontalcomponent of the movements. The bottom panel plots the verticalcomponent of the movements. The numbers indicate the different targetlocations of FIG. 2 . The circles represent the median of the player'smovements, across a plurality of repeated trials for each targetlocation. The curves represent the best-fit model of each movement.

FIG. 4 shows the variability of the movement time courses correspondingto the movement trajectories of FIG. 2 . The top panel of FIG. 4 plotsthe variability of the horizontal component of the movements. The bottompanel plots the variability of the vertical component of the movements.The eight curves in each panel correspond to the different targetlocations of FIG. 2 . In this example, variability was computed using arobust measure of the standard deviation (the median of the squareddeviation from the median).

FIG. 5 illustrates a method of assessing one or more metrics of a playerof a game, in accordance with an exemplary embodiment. As stated above,the metrics of a player can include speed, accuracy, precision, and/orreaction time. Accordingly, in the exemplary method, at step 502, themeasured metrics are sorted according to one or more goals of a game ora stage of the game (e.g., different target locations, durations, and/orsizes). At step 504, the median of the measured metrics are computedacross one or more repeated trials, separately for each goal. Forexample, the median of the player's movement trajectories is computedacross one or more repeated trials, separately for each goal. Exemplarymedian movement trajectories are displayed as the small dots in FIG. 2and as the circles in FIG. 3 . At step 506, each of the measured metricsis determined by using a custom mathematical model. For example, each ofthese median movement trajectories is fit with a mathematical model. Themovement trajectories can be modeled as having two component movements:an initial movement that ends near the location of a target followed bya corrective movement that ends closer to the target. These twocomponents are evident in the examples displayed in FIGS. 2 and 3 . Forthe movement trajectories displayed in FIGS. 2 and 3 , the player'sinitial movements tend to be hypermetric, passing beyond the targets,and then the corrective movements are in the opposite direction backtoward the target. For other players, the initial movements could behypometric, falling short of the target. Other players might exhibithypermetric movements for some target locations and hypometric movementsfor other target locations. In some embodiments, the two componentmovements are modeled as two successive sigmoidal functions:

$\begin{matrix}{{a.{f\left( {{t;a},b,c} \right)}} = \frac{a}{1 + e^{b({t - c})}}} & \left\lbrack {{Eq}.1} \right\rbrack\end{matrix}$ $\begin{matrix}{{b.{x(t)}} = {{f\left( {{t;p_{1}},p_{3},p_{4}} \right)} + {f\left( {{t;p_{5}},p_{7},p_{8}} \right)}}} & \left\lbrack {{Eq}.2} \right\rbrack\end{matrix}$ $\begin{matrix}{{c.{y(t)}} = {{f\left( {{t;p_{2}},p_{3},p_{4}} \right)} + {f\left( {{t;p_{5}},p_{7},p_{8}} \right)}}} & \left\lbrack {{Eq}.3} \right\rbrack\end{matrix}$

Eq. 1 defines the sigmoidal function. The values of x(t) in Eq. 2represent a model of the horizontal component of the movement trajectoryfor each time sample (examples of which are shown in the top panel ofFIG. 3 ). The values of y(t) in Eq. 3 represent a model of the verticalcomponent of the movement trajectory for each time sample (examples ofwhich are shown in the bottom panel of FIG. 3 ). In some embodiments,the values of the parameters, p₁, p₂, p₃, p₄, p₅, p₆, p₇, and p₈, arefit to the median movement trajectories, separately for each targetlocation. The best-fit parameter values are determined by using theLevenberg-Marquardt algorithm. The circles in FIG. 2 and curves in FIG.3 are examples of models of the movement trajectories, as expressed byEq. 1, with best-fit values for the parameters.

At step 508, the best-fit values of the parameters are used along withthe following equations to characterize each metric of the game. Forexample, the best fit values of the parameters are used to quantify theplayer's speed, precision, accuracy, and reaction time, separately foreach target location:

$\begin{matrix}{{a.{Speed}} = \frac{a_{m}}{f^{\prime}\left( {{p_{4};a_{m}},p_{3},p_{4}} \right)}} & \left\lbrack {{Eq}.4} \right\rbrack\end{matrix}$ $\begin{matrix}{{b.{Precision}} = \frac{v\left( p_{4} \right)}{a_{t}}} & \left\lbrack {{Eq}.5} \right\rbrack\end{matrix}$ $\begin{matrix}{{c.{Accuracy}} = \frac{e^{T}u}{a_{t}}} & \left\lbrack {{Eq}.6} \right\rbrack\end{matrix}$ $\begin{matrix}{{{d.{Reaction}}{Time}} = {p_{4} - {\frac{1}{2}{Speed}}}} & \left\lbrack {{Eq}.7} \right\rbrack\end{matrix}$

The function ƒ′(t) in Eq. 4 is the derivative of the sigmoidal function:

e. ƒ′(t;a,b,c)=bc[ƒ(t;1,b,c)][1−ƒ(t;1,b,c)]  [Eq. 8]

The value of a_(m) in Eq. 4 represents the amplitude of the first of thetwo component movements:

f. a _(m)√{square root over ((p ₁)²+(p ₂)²)}  [Eq. 9]

The value of a_(t) in Eq. 5 represents the distance to the targetlocation:

g. a _(t)=√{square root over ((x _(t))²+(y _(t))²)}  [Eq. 10]

where (x_(t), y_(t)) is the target location. The values of v(t) in Eq. 5represent the variability of movement trajectory (examples of which areshown in FIG. 4 ), combined across the horizontal and verticalcomponents of the movement. The vector e in Eq. 6 represents themovement error:

h. e(p ₁ −x _(t) ,p ₂ −y _(t))  [Eq. 11]

Finally, the vector u in Eq. 6 represents a unit vector in direction ofthe target location:

$\begin{matrix}{{i.u} = \frac{\left( {x_{t},y_{t}} \right)}{\left( {x_{t},y_{t}} \right)}} & \left\lbrack {{Eq}.12} \right\rbrack\end{matrix}$

In some embodiments, speed is quantified (Eq. 4) as movement durationand has units of milliseconds. Precision is quantified (Eq. 5) as thevariability of the movement trajectory, at the midpoint of the movement,scaled by the distance to the target location and has units of percent.Accuracy (Eq. 6) is quantified as the error of the initial component ofthe movement and has units of percent. Positive values for accuracyindicate that the movements are hypermetric whereas negative valuesindicate that the movements are hypometric. Reaction time (Eq. 7) isquantified as the midpoint of the movement minus half the movementduration and has units of milliseconds. At step 510, the outcome of eachmetric is combined across each goal of the game. For example, theoutcome for speed, precision, accuracy, and reaction time are combinedacross target locations by averaging across target locations. At 512,the outcome for each metric is converted to percentile score, such as bycomputing the player's performance against others who have played thegame. For example, the values for speed, precision, accuracy, andreaction time are converted to percentile scores, by comparing theplayer's performance with others who have played the game. A person ofordinary skill in the art would recognize that a variety of differentcomputations could be used to compute speed, accuracy, precision, andreaction time from the player's movements. For example, precision couldbe quantified as the average of the variability of the movementtrajectory, averaged across the duration of the movement (in addition toor instead of the variability at the at the midpoint of the movement asexpressed in Eq. 5).

As stated above, the game can assess one or more metrics. According toan embodiment, the game can determine the accuracy, speed, and precisionof the player's shots. Accuracy can be computed as a hit rate (e.g., theproportion or percentage of targets that are hit) separately for eachtarget location, target size, and/or target duration, and can beaveraged across locations, sizes, and/or durations. Speed can becomputed as the average amount of time it takes to hit the targets(e.g., the time interval from target presentation until it is hit),separately for each target location and/or target size, and averagedacross locations and/or sizes. Speed can also be characterized as thedistribution or cumulative distribution of such time intervals (i.e.,the cumulative hit rate over time). Precision can be characterized asthe distribution of shot locations relative to each target location.

According to another embodiment, the game can determine gain, gainvariability, spatial bias, kills per sec, time per kill, lapse rate,tracking accuracy, consistency, flick accuracy, and efficiency. Gain canbe computed as the distance of the player's input device movementdivided by the distance to the target from the initial input devicelocation before the movement. Gain variability can be computed as thevariance of the gain, across a plurality of targets. Spatial bias can becomputed as the mean and/or variance of any given metric (e.g.,accuracy, reaction time, etc.) as a function of location in a gamescenario. Kills per sec can be computed as the total number of targetsdestroyed divided by the amount of time (e.g., in seconds) that theplayer performed a training task. Time per kill can be computed as theamount of time (e.g., in seconds) from a target appearing until theplayer kills the target. Lapse rate can be computed as the number oftargets to which the player failed to respond divided by the totalnumber of targets presented. Tracking accuracy can be computed as thenumber of accurate shots divided by the total number of shots in tasksthat require smooth pursuit (rather than ballistic) movements to hit atarget. Consistency can be computed as the variance in the distributionof any given metric (e.g., accuracy, reaction time, etc.), across aplurality of targets. Flick accuracy can be computed as the number ofaccurate shots divided by the total number of targets presented in tasksthat require ballistic (rather than smooth pursuit) movements to hit atarget. Efficiency can be computed as the total number of accurate shotsdivided by the total number of shots plus the player's lapse rate,wherein the lapse rate is as computed as described above.

The game can comprise one or more scenarios for assessing a metric.According to an embodiment, the scenarios can relate to visual-detectionreaction time, auditory spatial-localization accuracy, change detectionaccuracy, and decision accuracy. In the visual-detection reaction timescenario, the player must respond as quickly as possible to targetspresented at randomized locations in a virtual environment. Targets arevisible for as long as it takes for the player to respond. This scenariomeasures how quickly a player can recognize a new object in a virtualenvironment. In the auditory spatial-localization accuracy scenario,targets are presented 360 degrees around the player in either random orequidistant patterns in a virtual environment. One of the targets emitsan audible cue, signaling the player to shoot the target that emittedthe audible cue. This scenario measures how well a player can spatiallylocalize an auditory cue. In the change detection accuracy scenario,targets are presented randomly in front of the player for a brief amountof time before they disappear. After a short duration, the same targetsreappear with one target in particular having a feature change (e.g.,different color, different spatial position, etc.). The player mustshoot the target with the feature that has been changed. This scenariomeasures the player's short-term memory and cognitive capacity. In thego/no-go scenario, a target of a particular color is presented for theplayer to shoot. Once hit, the target disappears and a new targetrandomly appears. The new target is the same or different color as theprevious target. The player is given a rule to follow, either to shootor ignore the second target if the second target color matches the firsttarget. Depending on a given rule, the player must decide to eithershoot the new target or ignore it. This scenario measures the player'sability to follow rules, make decisions, and inhibit unnecessaryresponses.

FIG. 6 illustrates a screen shot of a scorecard presented to a playerafter a round of a target practice game. The left side of the scorecardcomprises a plurality of dots corresponding to the locations of targetsthat a player attempted to shoot. The dots can be of different colorsthat indicate shot speed and/or accuracy at each target. The dots canalso be of different sizes that correspond to number of targetspresented at each location. For example, green dots could be used toindicate locations where the player was fast and accurate (e.g., theplayer was able to hit the small targets at those locations even whenthe targets were presented only for brief periods of time). Red dotscould be used to indicate locations where the player was slow andinaccurate (e.g., the player was unable to hit targets at thoselocations unless the targets were large and also presented forrelatively long periods of time). Orange and yellow dots could be usedto indicate intermediate levels of performance such that performance isworst at locations indicated by red dots, and progressively better fororange dots then yellow dots and then green dots. The right side of thescorecard comprises numerical representations of performance assessment.The circles in the top-right of the score card show percentile scoresfor accuracy, reaction time, and overall performance in comparison toother players. The left column lists some performance statistics for thecurrent practice session and the right column lists some performancestatistics for this player's best practice session.

FIG. 7 illustrates a screen shot of a scorecard that summarizes aplayer's performance for multiple rounds of a practice game. The upperleft portion of the screen shot displays the player's name, avatar, aswell as their composite score rating. Below this, separate compositemetrics are displayed and labeled (i.e., tracking accuracy, efficiency,etc.). The right portion of the screen shot displays short descriptivesummaries of observations made from the player's data across allmetrics. For instance, in this screen shot, there is a description of“Lower Screen Weakness” that explains to the player that they are lessaccurate when responding to targets in the bottom half of the screencompared to the top half of the screen.

FIG. 8 illustrates a screen shot from a target practice game whichcompares a player's performance with others who have played the game.The x-axis represents the composite skill score shown in FIG. 7 . They-axis represents the proportion of players with that composite skillscore. This allows the player to see the overall distribution ofcomposite skill scores across a population of players, and to see wherethey fall in that distribution. Bins of scores can be broken down intoskill tiers (e.g., bronze, silver, gold, etc.), as shown at the top ofthe screen shot.

FIG. 9 illustrates a screen shot from a target practice game thatdepicts a player's improvement over time. Players can sort their data byweapon (left side of the screen) and by bins of time (top of thescreen—daily, weekly, monthly, annually). The graph displays a givenmetric of interest chosen by the player (e.g., accuracy, reaction time,etc.), with additional data displayed below such as achievements ormilestones attained by the player.

FIG. 10 shows a screen shot from a zombie apocalypse game. The player'sgoal is to shoot each zombie in the head while the zombies are movingthrough a 3D virtual environment to attack the player. A person of skillin the art recognizes that a zombie can be stopped only by shooting itin the head. If a zombie reaches the player before being shot in thehead, then the round is over. Difficulty can be automatically adjustedfrom round-to-round based on the player's performance, and can beadjusted by changing the number of zombies and the manner in which theymove (speed, direction, and predictability of motion). For example, azombie running directly toward the player is relatively easy to hit, azombie moving laterally is more difficult to hit, and a zombie runningquickly on a serpentine path zigzagging toward the player, withreversals in direction at unpredictable times, is even more difficult tohit. Feedback can also be provided to train the player and improve theirperformance. This can occur from the player's interaction with the game.For example, blood spurts from the zombie's head when it is hit in thehead, and the spatial distribution of spurting blood indicates theaccuracy of the shot. Along these lines, if the player shots the zombiein the center of its head, then blood spurts symmetrically in alldirections, and if the player clips the right or left of the head, thenthe blood spurts asymmetrically to that direction.

One embodiment is a system comprising a computer and computer programthat implements a game (e.g., an eSport game) configured to assess aplayer's ability and to train the player to improve their performance.The computer program can present sensory stimulation to the player via adisplay that corresponds to a three-dimensional virtual environment. Thecomputer program also receives input signals from one or more inputcontrollers, operated by a player, to control the player's movements inthe virtual environment. The computer program changes the state of thevirtual environment based on the input signals, and re-renders thesensory stimulation dynamically over time according to the changes ofstate of the virtual environment. The computer program, furthermore,evaluates the input signals from the input controller to assess theplayer's performance. The computer program then dynamically manipulatesthe sensory stimulation and/or the evaluation of the input signals totrain the player to improve their performance.

In some embodiments, the eSport is a first-person shooter (FPS) eSportlike Overwatch. In other embodiments, the eSport is a multiplayer onlinebattle arena (MOBA) eSport like Star Craft or League of Legends. Aperson of skill in the art recognizes that other eSports or otherclasses of eSports could be substituted, including those that have notyet been reduced to practice.

In some embodiments, an individual plays the eSport by themselves. Inother embodiments, multiple people, teams of players and opponents, playat once.

The sensory stimulation includes visual, auditory, and/or somatosensorystimulation. Visual images can be rendered on a computer monitor, alaptop display, a video projector, a mobile device, a virtual realitydisplay or headset, or an augmented reality display or headset. Sensorystimulation can also be rendered with brain-computer interface (alsocalled brain-machine interface) methods, devices, apparatuses, orsystems for stimulating neural activity, including, but not limited to,retinal prostheses and cochlear implants. A person of skill in the artrecognizes that other devices, apparatuses, or systems for presentingvisual stimulation could be substituted, including those that have notyet been reduced to practice. Likewise, a person of skill in the artrecognizes that there are a variety of devices or apparatuses forpresenting auditory or somatosensory stimulation, and a person of skillin the art recognizes any such devices, apparatuses, or systems could besubstituted, including those that have not yet been reduced to practice.

Examples of input controllers include keyboard, mouse, gaming mouse,video game console, joystick, accelerometer, pointing device, motioncapture, Wii remote controller, eye tracker, computer vision system, orany of a variety of methods, devices, apparatuses, and systems forsensing, measuring or estimating human movement to provide input signalsto a computer. Examples of human movements include hand movements, armmovements, head movements, body movements, and eye movements. Examplesof input controllers also include brain-computer interface methods,devices, apparatuses, and systems for sensing, measuring or estimatingbrain activity. Examples of brain-computer interfaces include, but arenot limited to, devices that measure electrophysiogical signals (e.g.,using EEG, MEG, microelectrodes) and optical signals (e.g., usingvoltage-sensitive dyes, calcium indicators, intrinsic signals,functional near-infrared spectroscopy, etc.). Examples of brain-computerinterfaces also include other neuroimaging techniques (e.g., functionalmagnetic resonance imaging). Examples of input controllers also includemethods, devices, apparatuses, or systems for sensing, measuring orestimating physiological data. Physiological data includes, but is notlimited to, EEG, EKG, EMG, EOG, pupil size, and biomechanical datarelating to breathing and/or respiration. A person of skill in the artrecognizes that any such input controller or any combination of suchinput controllers could be used. It is also recognized that othermethods, devices, apparatuses, or systems for sensing, measuring orestimating human movement or physiological activity could besubstituted, including those that have not yet been reduced to practice.

Input signals include digital input codes (e.g., ascii keyboard codes,mouse clicks) or analog electrical signals. The input signals could beprovided to the computer via electrical (e.g., USB) or wirelessinterface.

The virtual environment is either two-dimensional (2D) orthree-dimensional (3D). The state of virtual environment includes, butis not limited to, the 2D or 3D position and orientation of theviewpoint of the player, the 2D or 3D position and orientation of thevirtual character or agent being controlled by the player, the 2D or 3Dposition and orientation of the characters being controlled byteammates, the 2D or 3D position and orientation of objects in thevirtual environment, the 2D or 3D position and orientation of virtualopponents, the status (e.g., health) of the virtual character beingcontrolled by the player, the status of virtual teammates, and thestatus of virtual opponents. The state of the environment alsooptionally includes factors that determine the simulated movements ofobjects in the virtual environment (e.g., force, mass, gravity,friction). A person of skill in the art recognizes that the system couldperform an accurate physical simulation of the virtual environment inaccord with the physics of motion. It is also recognized that the systemcould simulate an alternative physics of motion that does not correspondto such a real environment. In addition, the state of the environmentoptionally includes lighting and factors that determine the rendering ofsensory stimulation (e.g., field of view for rendering images). A personof skill in the art recognizes that the system could perform an accuratephysical simulation of the virtual environment in accordance with thephysics of light (for visual stimulation), sound (for auditorystimulation), and pressure (for somatosensory stimulation). It is alsorecognized that the system could simulate an alternative physics oflight, sound, or pressure that does not correspond to such a realenvironment.

Each player interacts with the virtual environment. Examples of theseinteractions include moving (e.g., changing position, pose, orientation,etc.) characters being controlled by each player, and/or moving otherobjects in the virtual environment, via an input controller. Examples ofmoving other objects in the virtual environment include picking-up,carrying, and putting-down objects, firing projectiles, etc.

Assessing performance optionally includes, but is not limited to,measuring the speed and/or accuracy of a player's movements. Accuracycan be measured in units of distance or as a hit rate (e.g., apercentage or proportion of targets hit). Speed can be measured in unitsof time (e.g., milliseconds) or distance per unit time (e.g., miles perhour). Assessing performance can also optionally include measuringprecision and reaction time. Assessing performance can also optionallyinclude measuring a player's speed-accuracy tradeoff curve. Assessingperformance can also optionally include measuring the relative value ofa series of movements executed by a player. The relative value of aseries of movements depends on the objective goals of the game. Forexample, the “Cornershot/Pentakill” scenario described above, requires aplayer to make a series of movements to shoot a plurality of targets andperformance can be measured as the hit rate. Assessing performance canalso optionally include measuring visual-detection reaction time,auditory spatial-localization accuracy, change detection accuracy,and/or accuracy in decisions about whether or not to perform an actionin the game (e.g., shoot). Assessing performance can also optionallyinclude measuring movement gain, gain variability, spatial bias, actionsper second (e.g., kills per second), time per action (e.g., time perkill), lapse rate, tracking accuracy, consistency, flick accuracy,and/or efficiency. Assessing performance can also optionally includemeasuring decision-making abilities in flexible contexts, visual acuity,memory, learning rate, cognitive control to ignore distractors, and/orrate of adaptation. Assessing performance can also optionally includemeasures that are established in the prior art and in the eSports andvideo game industry, including kill-death ratio, damage dealt, damageaccrued, damage blocked, time spent on objective, kills or deaths byobjective, critical damage, healing dealt, healing accrued, assists,and/or final blows. Kill-death ratio can be the total number of enemieskilled by a player divided by the number of times enemies kill thatplayer. Damage dealt can be a weighted hit rate. The weight can dependon the weapon used by the player. Damage dealt can be the sum of theplayer's accurate shots to enemies, and can be each multiplied by theweapon-dependent weight. Damage accrued can be the number of accurateshots enemies have landed on a player, and can be each multiplied by aweapon-dependent weight. Damage blocked can be the amount of damageenemies have attempted to inflict on a player that was blocked by theplayer using a shield or skill that absorbs enemy damage, preventing itfrom being applied to the player. Time spent on an objective can be theamount of time (e.g., in seconds) a player spends within the boundariesof a given objective (e.g., remaining in a particular spatial positionof a map). Kills or deaths by objective can represent the number oftimes enemies have killed the player or number of enemies the player haskilled while the player and enemies were located within the boundariesof a given objective. Critical damage can represent the total amount ofdamage a player has inflicted on an enemy by landing shots in a criticalarea, such as the head of an enemy (i.e., a headshot). Healing dealt canrepresent the total amount of healing a player has applied to teammates.Healing accrued can represent the total amount of healing a player hasreceived from teammates. Assists can represent the number of enemies aplayer has dealt damage to, regardless of whether the player alsodelivered the final amount of damage that killed an enemy (i.e., finalblow). Final blows can represent the total number of enemies for whichthe player dealt the final amount of damage that killed an enemy.

For input controllers that sense or measure physical movements (e.g.,mouse, keyboard, eye tracker, motion capture, etc.), there are variousmethods for quantifying the speed of a player's movements. In oneembodiment, speed can be quantified as the duration of time between theonset of a movement and the termination of the movement. In anotherembodiment, speed can be quantified as the duration of time between theappearance of a target and the termination of a movement.

Human movements typically have multiple different stages. For example, amovement typically consists of an initial movement that lands near thelocation of a target followed by one or more corrective movements thatland closer to the target. In one embodiment, speed can be quantified asthe durations of each such component movement. In another embodiment,speed can be quantified as the entire duration of the initial movementplus the durations of one or more of the corrective movements. Inanother embodiment, speed can be quantified as the entire duration of aseries of actions and decisions to attain a goal, for example,completing an in-game objective with a series of movements. In anotherembodiment, speed can be quantified as the amount of time it takes toachieve an objective. For example, speed can refer to the time intervalfrom target presentation until it is hit. In another embodiment, speedcan be characterized as the distribution or cumulative distribution ofsuch time intervals. For example, speed can be characterized as thecumulative hit rate over time. Speed can be measured separately for eachtarget or combined (e.g., by averaging) across multiple targets.

For input controllers that sense or measure physical movements (e.g.,mouse, keyboard, eye tracker, motion capture, etc.), there are variousmethods for quantifying the accuracy of a player's movements. As notedabove, each such input controller senses, measures, or estimates humanmovement and provides input signals to a computer to interact with thevirtual environment. For example, the input signals could change 2D or3D position and orientation of the virtual character being controlled bythe player, or the 2D or 3D position and orientation of a virtual objectin the virtual environment of the game. In one embodiment, the inputsignals control a cursor that is rendered on a computer monitor. Theaccuracy of a player's movements can be quantified either in terms ofthe physical movement in the real environment (e.g., the position orrelative position of the mouse), or in terms of the virtual movement inthe virtual environment (e.g., the position of a cursor that iscontrolled by the mouse). For example, the 2D screen position of acursor on a computer monitor can be compared to the 2D projection of the3D location of an object in the virtual environment. To do so, thecomputer converts a 3D location in the virtual environment to thecorresponding 2D screen position that is the projection of that 3Dlocation. In another embodiment, accuracy can be quantified in terms ofthe 2D or 3D position and orientation of a virtual object in the virtualenvironment. As noted above, human movements typically have multipledifferent stages. In one embodiment, the accuracy of the initialmovement can be quantified. In another embodiment, the accuracy of thefinal end point after one or more of the corrective movements can bequantified together with the original movement. In another embodiment,the accuracies of each component movement can be quantified. Theaccuracy of an individual movement can be quantified as the errorbetween the executed movement and the desired movement, for example, thedifference between the end point of a cursor movement and the positionof a target on the computer display. The accuracy includes the magnitudeand/or direction of the error. In one embodiment, the distribution ofsuch errors can be measured and characterized. The distribution oferrors can be characterized by computing statistics from a plurality ofsuch errors. Statistics that can be computed include mean, median, mode,variance, covariance, skewness, kurtosis, and higher order statisticalmoments. In another embodiment, the distribution of errors can becharacterized by fitting a model to the plurality of such errors. Thedistribution of errors can be fit with a statistical model (e.g., amultivariate normal distribution). In another embodiment, thedistribution of errors can be fit by a functional model (e.g., a modelof the neural processing that controls eye movements or body movements).It is also recognized that various different statistical or functionalmodels can be substituted, including those that have not yet beenreduced to practice. Accuracy can be measured separately for each targetor combined (e.g., by averaging) across targets.

Assessing performance can include the precision of a player's movements.According to an embodiment, precision can be quantified as theconsistency of a movement when it is repeated, for example, when atarget is presented at the same location multiple times. Consistency canbe quantified as the standard deviation, across repeats, of the 2D or 3Dpositions and/or orientations of a virtual character being controlled bythe player, or of the 2D or 3D positions and/or orientations of avirtual object in the virtual environment. Other statistics, in additionto standard deviation, can also be used to quantify precision, such asmean, median, mode, variance, covariance, skewness, kurtosis, and higherorder statistical moments. Various different statistical or functionalmodels can be substituted, including those that have not yet beenreduced to practice. According to another embodiment, precision can becharacterized as the spatial distribution of errors, i.e., the spatialdistribution of shot locations relative to a target location. Forexample, the spatial distribution of errors can be fit with astatistical model (e.g., a multivariate normal distribution), andprecision can be quantified as the variance and covariance of thebest-fit normal distribution. It is also recognized that variousdifferent statistical or functional models can be substituted, includingthose that have not yet been reduced to practice. Precision can bemeasured separately for each target or combined (e.g., by averaging)across targets.

Assessing performance can also include the reaction time of a player'smovements.

Assessing performance can be measured with physiological signals (e.g.,brain-computer interface, EEG, EMG, etc.) from one or more inputdevices, as noted above. In one embodiment, the input signals from aninput controller can be used to control a cursor that is rendered on acomputer monitor, and the speed, precision, accuracy, and reaction timeof a player's movements are quantified in terms of the position of acursor. In another embodiment, the input signals from such an inputcontroller can be used to control the 2D or 3D position and orientationof a virtual object in the virtual environment, and the speed,precision, accuracy, and reaction time of a player's movements arequantified in terms of the virtual movement in the virtual environment.The preceding paragraphs describe several embodiments with variousmethods for quantifying speed, precision, accuracy, and reaction time.Each of those methods for quantifying speed, precision, accuracy, andreaction time can be embodied with input controllers that measure eitherphysical movements (e.g., mouse) or physiological signals (e.g., EMG).Various different statistical or functional models can be substitutedfor measuring speed, precision, accuracy, and/or reaction time fromphysiological signals, including those that have not yet been reduced topractice.

Assessing performance can also include analyzing a player'sspeed-accuracy tradeoff. Typically, human behavior exhibits aspeed-accuracy tradeoff. Faster movements are typically less accurate,and more accurate movements are typically slower. FIG. 11 illustratesexemplary speed-accuracy tradeoff curves. The points A and B are on thesame speed-accuracy tradeoff curve for a given player. The point labeledA corresponds to faster but less accurate performance compared to thepoint labeled B. The point labeled C is on a different speed-accuracytradeoff curve. The curve that includes the point labeled C correspondsto a player that has overall better performance than the curve thatincludes the points labeled A and B.

Examples of improved performance include making faster and/or moreaccurate movements, making a more valuable series of movements versusalternative less valuable movements, making movements that correspond toa better speed-accuracy tradeoff curve, and making movements thatcorrespond to a better tradeoff between speed and accuracy. In oneembodiment, performance improvement can be indicated as a transitionfrom one speed-accuracy tradeoff curve (e.g., the curve in FIG. 11 thatincludes points A or B) to a different speed-accuracy tradeoff curve(e.g., the curve in FIG. 11 that includes point C). In otherembodiments, performance improvement can be indicated by any of themeans described above for assessing performance.

One embodiment provides a service to help players choose teammatesand/or to help teams and coaches identify and recruit talented players.Performance can be assessed for each of a plurality of players. Theperformance assessment can be shared via either a website or socialnetwork. The website or social network can provide a platform forplayers with complementary abilities to form teams and for coaches tochoose players for particular positions on a team, based on the players'performance assessments. For example, in a game that has specificcharacter classes or team roles (e.g., Overwatch), a first player withexcellent decision-making skills but poor accuracy skills may be bestcomplemented by playing with a second player with excellent accuracyskills, as each would fill specific team or class roles. This preventsteams from being constructed of players with skillsets with too muchoverlap leading to holes or weaknesses in team composition. Anotherexample, in the case of games without specified class or team roles(e.g., Counterstrike Global Offensive), is a first player with highaccuracy being matched with a second player of equal or similar accuracyskill, as mismatches in talent cause poor team performance and poorergaming experiences.

In some embodiments, the computer program dynamically manipulates thesensory stimulation to train a player to improve their performance. Forexample, in a first-person shooter eSport game where a player's goal isto hit targets, the targets can be automatically placed at locationswhere the player is slow and/or inaccurate. The target locations caninclude 2D screen positions or 3D locations in the virtual environment.

In one embodiment, the targets are presented for a limited period oftime and the target presentation duration is manipulated, depending onthe history of the player's performance accuracy, separately for eachlocation (2D screen position or 3D location in the virtual environment).The target presentation duration is randomized or pseudo-randomized overa range of different possible durations. Each possible duration canselected independently for each target location, and can be adjusteddynamically over time as the player's performance improves. In oneembodiment, the target presentation duration is manipulated by anadaptive staircase procedure in which the target presentation durationfor a particular location is decreased after at least one hit at thatlocation and the target presentation duration is increased after atleast one miss at that location. In other embodiments, the targetpresentation duration is manipulated by the QUEST or BEST procedures. Aperson of skill in the art recognizes that other adaptive procedures ormethods for manipulating target presentation duration could besubstituted including those that have not yet been reduced to practice.

In one embodiment, the targets are presented at particular locations,depending on the history of the player's performance accuracy. Targetlocations on the screen (e.g., 2D coordinates) are determined by theindividual player's performance in those screen locations. For example,if a player performs poorly in the upper-left portion of the screencompared elsewhere, then more targets will be presented in theupper-left portion of the screen relative to the distribution of targetselsewhere. The weighting of this distribution can be dependent upon thelevel of disparity in performance between locations. This distributioncan be updated dynamically between training tasks to adapt the spatialdistribution of targets for each new training task performed by theplayer.

Some embodiments provide the player with feedback about theirperformance. Examples of feedback include visual features or animationsrendered as part of the virtual environment or superimposed on therendered images of the virtual environment. These visual features oranimations can be presented during or immediately following a player'smovements. Examples of feedback also include cross-hairs or coloredspots showing where a movement was made (e.g., where a shot was fired)in comparison to where it should have been made.

One embodiment comprises explosive feedback for when a player performsan objective or task. The explosive feedback can be indicative of howwell the player performed the objective or task. For example, if theplayer hits the target dead center of an object, then the objectexplodes symmetrically in all directions. However, if the player clipsthe edge of the object, then the explosion of the object is asymmetricalin that direction. Moreover, if the player clips the right or left earof a target, then blood spurts out to the right or left, respectively.Any of various different spatial distributions for the explosivefeedback could be used.

Another embodiment comprises corrective feedback based on a player'saction toward an objective or task. Corrective feedback can be used totrain a player to improve their performance. For example, a target canbe removed when a player initiates a movement toward the target. Thisforces the player to make a ballistic movement toward the rememberedlocation of the target. The target is re-presented after the playercompletes a movement or fires one or more shots, thereby providingfeedback to initiate a corrective movement. Due to the feedback andcorrective movement, the player's initial ballistic movement willimprove (faster and/or more accurate) over time with practice.

In another embodiment, feedback is a visual representation summarizing aplayer's performance over a period of time, e.g., indicating locationson the screen where movement speed and/or accuracy was better or worse.

Feedback also optionally includes auditory tones or sounds presentedduring or immediately following a player's movements. Feedback alsooptionally includes somatosensory stimulation.

Some embodiments include measuring a player's speed-accuracy tradeoff,and then training the player to improve their performance by learning tocontrol their speed-accuracy tradeoff. For example, when playing againsta fast opponent, a player should choose to be as fast as possible, eventhough they will typically be less accurate (e.g., corresponding topoint A in FIG. 11 ). When playing against a slower but more accurateopponent, on the other hand, a player should choose to be slower andmore accurate (e.g., corresponding to point B in FIG. 11 ). Oneembodiment is a first-person shooter eSport game in which a player'sgoal is to hit targets. Different targets (e.g., different colors suchas red, yellow, and green) cue the player to be as fast as possible oras accurate as possible or various options in between, and the targetsshoot back at the player with different latencies. The cue meaning “beas fast as possible” (e.g., green) fires back at the player with a shortlatency but with low accuracy, and the cue meaning “be as accurate aspossible” (e.g., red) fires back at the player with long latency butvery accurately. This enables measuring several samples (one for each ofthe different cues) on the speed-accuracy tradeoff curve. Over training,a player's entire speed-accuracy tradeoff curve can improve, shifting upand to the right in the graph illustrated in FIG. 11 . In addition, thisembodiment trains players to optimize where they choose to be on thespeed-accuracy tradeoff curve for each target. Once a player is welltrained, they will be able to choose on the fly to optimize theirspeed-accuracy depending on which opponent they are up against during acompetition. A person of skill in the art recognizes that otherprocedures or methods for cueing players to trade off speed and accuracycould be substituted including those that have not yet been reduced topractice.

Some embodiments include training a player to be less prone todistraction. Sensorimotor performance depends on attention and aperson's attention can be diverted by distracting stimuli. For example,a player's performance (e.g., speed and/or accuracy) in hitting a targetat one location could be impaired by flashing a distracting non-targetstimulus at a different location just before the target is presented.One embodiment is a target practice game with distractors. Targets anddistractors are presented at various locations in the virtualenvironment. The visual appearance of targets and distractors differfrom one another (e.g., color, shape, etc.). A player's goal is to hitthe targets, and the player's performance is assessed by measuring speedand accuracy. The distractors are presented in advance of the targets,and the stimulus onset asynchrony (SOA, the interval of time between thedistractor onset and the target onset) is manipulated, depending on thehistory of the player's performance accuracy, separately for each targetlocation and for each distractor location (2D screen position or 3Dlocation in the virtual environment). The SOA is randomized orpseudo-randomized over a range of duration values, the range of SOAduration values is selected independently for each target location andeach distractor location, and the range of SOA duration values isadjusted dynamically over time as the player's performance improves. Aperson of skill in the art recognizes that various adaptive proceduresor methods for manipulating SOA duration could be used including thosedescribed above, as well as those that have not yet been reduced topractice.

In some embodiments, the computer program dynamically manipulates theevaluation of input signals from an input device to train a player toadjust their movements of the input device accordingly. In oneembodiment, a player's performance is improved by using a motoradaptation protocol to train the player to make faster and/or moreaccurate movements. In one embodiment, the mapping from mouse positionto screen position is adjusted dynamically during training. For example,if a player's movement to a particular target location is hypometricthen the mouse movements toward targets at that location are remapped soas to make the movement more hypometric. The player can thenautomatically learn to compensate for such changes, leading to improvedperformance accuracy. Other embodiments include manipulating theposition, velocity, acceleration, or higher-order temporal derivativesof position acquired with one of the input controllers. Otherembodiments include manipulating the orientation, angular velocity,angular acceleration, or higher-order temporal derivatives oforientation acquired with one of the input controllers. Otherembodiments include manipulating the joint angle, angular velocity ofjoint angle, angular acceleration of joint angle, or higher-ordertemporal derivatives of joint angle acquired with one of the inputcontrollers.

Other embodiments include assessing performance of a player as a part ofa team. For example, multiple players, each assigned a specific combatrole, can cooperate to accomplish one or more structured challenges,such as surviving an onslaught of enemy zombies. One player's role canbe to damage and eliminate as many zombies as possible, while the otherplayer's role can be to use a temporary shield or healing mechanic toblunt or heal damage caused by zombie enemies. The overall goal of thechallenge is to survive as long as possible while zombie enemies attackboth players in perpetuity. Performance can be measured by how long theteam survives. Player performance can be assessed within the context ofeach player's assigned role, and can be modulated by the performance oftheir teammate. As such, the damage output of each enemy zombie ismanipulated to increasingly tax and train the performance of the playerassigned to heal or shield the other player, while the base amount ofhealth of each zombie is manipulated to tax and train the abilities ofthe player assigned to dealing damage. This is used to test and trainhow well individual players can adapt and thrive in cooperation withother teammates, and to measure how the skill level of a teammateimpacts an individual player's skill.

Additional embodiments are multiplayer objectives in which multipleplayers are assigned any combination of the following roles: damagedealer, healer, tank, or crowd control. The damage dealer uses a rangeof given weapons (e.g., guns, cybernetics, or magical powers) to inflictdamage and ultimately kill enemies. A healer's role is the opposite;rather than inflict damage upon enemies, they heal damage caused byenemies to players on the healer's team. This can be done by any rangeof weapons (e.g., guns, cybernetics, or magic) or skills (e.g.,cybernetics, magic). The goal of a tank is to shield teammates byabsorbing as much enemy damage as possible through either an increase inbase health, defensive tools (e.g., shields, armor), or powers (e.g.,magic, cybernetics). Crowd control roles support their team bycontrolling the movement or abilities of enemies through various methods(e.g., guns with special abilities, magic, cybernetics). For example, acrowd control player may fire a weapon that slows down the movement ortemporarily paralyzes an enemy combatant to prevent them from attackingthe player's teammates, or making them easier to kill.

Some embodiments include performance scores. The scores can reportdifferent aspects of performance (e.g., speed, precision, accuracy,reaction time at each location in the virtual environment). Each aspectof performance can be scored separately, or the various individualscores can be combined into overall scores (e.g., overall speed combinedacross all locations, or overall performance combined across speed,precision, accuracy, and reaction time, and combined across alllocations). The scores for different aspects of performance can bemeasured in different units. For example, speed can be measured in unitsof time (e.g., milliseconds), and accuracy can be measured in units ofdistance. To combine these disparate scores, some embodiments converteach performance score to a percentile score by comparing a player'sindividual performance with that of a plurality of other players. Then,the percentile scores can be further combined to compute an overallscore. In one embodiment, for example, a player's speed is convertedfrom units of time to a percentile and the player's accuracy isconverted from units of distance to a percentile, and then the twopercentile scores are averaged to compute an overall score. In someembodiments, a player can view a score card after each round. In someembodiments, a player can view visual representations of how their scorechanges over time with practice. In some embodiments, a player cancompare their scores with other players. Some embodiments can include aleader board that shows the scores of the best players.

Another embodiment is a system for characterizing and classifying thestrengths and weaknesses of players, by applying pattern recognition,pattern classification, or machine learning operations to analyzeperformance assessment data from a plurality of players and to comparethe performance assessment of an individual player with that of theplurality of players. Pattern recognition pattern classification andmachine learning operations can include correlation, canonicalcorrelation, sum of squared difference, least-squares, partial leastsquares, nearest neighbor, Mahalanobis distance, regression, multiplelinear regression, logistic regression, polynomial regression, generallinear model, principal components analysis (PCA), singular valuedecomposition (SVD), factor analysis, principal components regression,independent components analysis (ICA), multidimensional scaling,dimensionality reduction, maximum likelihood classifier, maximum aposteriori classifier, Bayesian classifier, Bayesian decision rule,radial basis functions, linear discriminant analysis, regularizeddiscriminant analysis, general linear discriminant analysis, flexiblediscriminant analysis, penalized discriminant analysis, mixturediscriminant analysis, Fischer linear discriminant, regularization,density estimation, naive Bayes classifier, mixture model, Gaussianmixtures, minimum description length, cross-validation, bootstrapmethods, EM algorithm, Markov chain Monte Carlo (MCMC) methods,regression trees, classification trees, boosting, AdaBoost, gradientboosting, neural network classifier, projection pursuit, projectionpursuit regression, support vector machine, support vector classifier,K-means clustering. vector quantization, k-nearest-neighbor classifier,adaptive nearest-neighbor classifier, cluster analysis, clusteringalgorithms, k-medoids, hierarchical clustering, sparse principalcomponents, non-negative matrix factorization, nonlinear dimensionreduction, undirected graph models, statistical learning, supervisedlearning, and unsupervised learning. Embodiments of this disclosure arenot limited to the pattern recognition, pattern classification, andmachine learning operations listed above, which are given as a subset ofthe pattern recognition, pattern classification, and machine learningoperations that can be applied to process the performance assessmentdata.

Another embodiment is a system comprising a computer and a firstcomputer program to assess a player's ability while they are playing aneSport or video game that is implemented in a second computer programthat is operating in parallel on the same computer. The first computerprogram receives input signals from input controllers to measure theplayer's movements. The first computer program optionally reads from thecomputer memory to determine the state of the second computer program(the eSport) including the locations of targets. The first computerprogram, furthermore, evaluates the input signals from the inputcontroller, to assess the player's performance while playing the eSport.

Other embodiments are cheater detection systems or methods. Some playerscheat by running a second computer program (a cheat program) that isoperating in parallel on the same computer as the first computer program(an eSport or a video game). The second computer program might, forexample, read from the computer memory to determine the state of thefirst computer program (the eSport) including the locations of targets,and write to memory in such a way as to automatically shoot the targetswithout the need for the player to move or manipulate their inputdevice. However, the second computer program does not mimic naturalhuman movements. One embodiment for a cheat detector is a method that isincorporated into the first computer program (the eSport or a videogame), and that compares the performance of a player with a database ofnatural human movements. A cheater is detected when the player'sperformance is inconsistent with natural human movements. Such cheatdetection systems or methods operate by applying pattern recognition,pattern classification, or machine learning operations to analyzeperformance assessment data from a large number of players. Some patternrecognition operations, pattern classification, and machine learning arelisted above.

FIG. 12 illustrates a exemplary method for detecting a cheater in agame. First, at step 1201, each movement in a database of playermovements for performing a task (e.g., shooting targets) is representedas a vector of numbers. For example, each number in each vector canrepresent the x- or y-coordinate of the 2D position of cursor controlledby a player using an input controller, such that the successive numbersin each vector can represent successive positions during a movement.Second, at step 1202, the vectors are processed to compute a principalcomponents analysis (PCA). Principal component analysis (PCA) is astatistical procedure that converts an input set of vectors to an outputset of vectors called principal components. The output set of vectors isan orthonormal basis for the input set of vectors. The projection (dotproduct) between any input vector and the principal components computesa vector of principal component scores. Third, at step 1204, a smallnumber of principal components are extracted from PCA that account for alarge proportion of the variance in the players' movements. The numberof principal components extracted can range from 1 to N where N is thelength of each input vector. But the number of principal componentsextracted is typically much smaller than N (for example, in the range 1to 10). Fourth, at step 1206, each individual movement is projected ontothe small number of principal components, thereby yielding alow-dimension representation of each individual movement. For example, 3principal components can be used to compute 3 principal component scoresfor each input vector, even though each input vector can comprisehundreds of numbers. Fifth, at step 1208, a new movement (that was notincluded in the original database of movements) is projected onto thesame principal components, yielding a low-dimensional representation ofthat new movement. Lastly, at step 1210, the low-dimensionalrepresentation of the new movement is compared with the distribution oflow-dimensional representations of the original database of movements todetermine if the new movement is an outlier, i.e., to classify the newmovement as a natural human movement or a cheater. Other embodimentsutilize other pattern recognition, pattern classification, or machinelearning operations in addition to, or instead of, PCA. Embodiments arenot limited to the pattern recognition, pattern classification, andmachine learning operations listed above, which are given as a subset ofthe pattern recognition, pattern classification, and machine learningoperations that can be applied to detect cheaters.

Another embodiment is a smart input controller that corrects for biasesin a player's movements. A person of skill in the art recognizes that aplayer's movements exhibit systematic biases. Some movement errors arerandom and best described by a statistical distribution. But othermovement errors are systematic biases. FIG. 13 , for example,illustrates an arm movement to reposition a mouse from point A towardpoint B. The arm movement is hypometric, falling short of the desiredtarget. The figure also illustrates an arm movement from point C towardpoint D that is hypermetric, overshooting the desired target. In oneembodiment, a player makes a series movements from each of a pluralityof starting positions to each of a plurality of target positions. Theposition error (e.g., the amplitude and direction of the differencebetween the landing position of the movement and the position of thetarget) is measured for each movement, and the average movement error iscomputed for each combination of a starting position and a targetposition. The computer program then manipulates the evaluation of theinput signals from the input controller to correct for systematic biasesin the player's movements. Another embodiment measures and corrects forsystematic biases in at least one of position, velocity, movementacceleration, and higher-order temporal derivatives of position. Anotherembodiment measures and corrects for systematic biases in at least oneof orientation, angular velocity, angular acceleration, and higher-ordertemporal derivatives of orientation. Another embodiment measures andcorrects for systematic biases in the joint angle of at least one joint.Another embodiment measures and corrects for systematic biases in atleast one of angular velocity of joint angle, angular acceleration ofjoint angle, and higher-order temporal derivatives of joint angle. Inone embodiment, the distribution of errors is characterized by fitting amodel to the plurality of such errors, fit with a statistical model(e.g., a multivariate normal distribution). In another embodiment, thedistribution of errors is fit by a functional model (e.g., a model ofthe neural processing that controls eye movements or body movements). Itis also recognized that various different statistical or functionalmodels could be substituted, including those that have not yet beenreduced to practice.

Another embodiment is a computer program that sets the sensitivity on aninput controller (e.g., the mouse sensitivity) that corrects for biasesin a player's movements. A player makes a series movements from each ofa plurality of starting positions to each of a plurality of targetpositions. The position error (e.g., the amplitude and direction of thedifference between the landing position of the movement and the positionof the target) is measured for each movement, and the average movementerror is computed for each combination of a starting position and atarget position. The computer program then manipulates the sensitivityof the input controller to correct for systematic biases in the player'smovements.

Referring to FIG. 14 , an exemplary process for assessing performance ofa player of a game is illustrated. Beginning at step 1402, an input fromthe player of the game can be received. Subsequently, at step 1404, theinput from the player during the game can be monitored. Next, at step1406, a characteristic of the game resulting from the input from theplayer can be determined. Thereafter, at step 1408, based on the inputfrom the player during the game, a performance of the player during aperiod of time in the game can be assessed. The performance of theplayer can be related to one or more metrics of the game. Moreover, theperformance of the relating to the player of can comprise comparing theinput of the relating to the player of during the period of time in thegame to an optimal input from the player during the period of time inthe game.

According to another embodiment, a method for sensorimotor assessment isprovided, comprising: (i) receiving information about at least onepersons' movements when interacting with a virtual environment; and (ii)monitoring at least one of the persons' movements to assess the persons'performance interacting with the virtual environment, whereinperformance comprises at least one of speed and accuracy of movement.The method can further comprise: (i) applying at least one of patternrecognition, pattern classification, and machine learning operations toanalyze performance assessment of the plurality of persons; and (ii)comparing the performance assessment of an individual player with thatof the plurality of players.

The performance assessment further comprises at least one of speed,precision, accuracy, reaction time, a speed-accuracy tradeoff, spatialbias, movement gain, gain variability, lapse rate, consistency,efficiency, tracking accuracy, flick accuracy, visual acuity,visual-detection reaction time, auditory spatial localization accuracy,change detection accuracy, decision accuracy, rate of adaptation,attention, cognitive control to ignore distractors, cognitive capacity,accuracy in decisions about whether or not to execute a movement,decision-making abilities, memory, learning rate, a relative value of aseries of movements, kills per sec, time per kill, kill-death ratio,damage dealt, damage accrued, damage blocked, time spent on objective,kills or deaths by objective, critical damage, healing dealt, healingaccrued, assists, and final blows.

The method can further comprise presenting distracting sensorystimulation to at least one of the persons.

The method can further comprise a service to help players chooseteammates, to automatically select teammates, or to help eSports teamsand coaches identify and recruit talented players, based the performanceassessments of at least one of the persons. The service can be at leastone of a website, a mobile app, an in-game overlay, or a social network.

According to another embodiment, a method for sensorimotor training isprovided, comprising: (i) presenting sensory stimulation to at least oneperson, wherein the sensory stimulation comprises at least one ofvisual, auditory, or somatosensory stimulation, and wherein the sensorystimulation is rendered to correspond to a virtual environment; (ii)receiving information about at least one of the persons' movements wheninteracting with the virtual environment; (iii) monitoring the movementsto assess at least one of the persons' performance interacting with thevirtual environment, wherein performance comprises at least one of speedand accuracy of movement; and (iv) manipulating the sensory stimulationpresented to at least one of persons based on the assessment of theirperformance, wherein the manipulation of sensory stimulation is designedto improve the persons' performance.

The manipulation of the sensory stimulation further comprises at leastone of placing target stimuli at locations where at least one of theplayer's performance is slow, placing target stimuli at locations whereat least one of the persons' performance is inaccurate, changing thecontrast of at least one target stimulus, changing the transparency ofat least one target stimulus, changing the size of at least one targetstimulus, changing the timing of presentation of at least one targetstimulus, changing the presentation duration of at least one targetstimulus, changing the color of at least one target stimulus, changingthe shape of at least one target stimulus, changing the texture of atleast one target stimulus, changing the temporal frequency of at leastone target stimulus, changing the speed of motion of at least one targetstimulus, and changing the direction of motion of at least one targetstimulus.

The improved performance further comprises at least one of making fastermovements, making more accurate movements, making more precisemovements, making movements with a shorter reaction time, makingmovements that correspond to a better speed-accuracy tradeoff curve,making movements that correspond to a better tradeoff between speed andaccuracy, making a more valuable series of movements versus alternativeless valuable movements, faster visual-detection reaction time, moreaccurate auditory spatial-location, more accurate change detection,better accuracy in decisions about whether or not to execute a movement,more accurate movement gain, less gain variability, less spatial bias,more kills per sec, less time per kill, lower lapse rate, more accuratetracking, greater consistency, better flick accuracy, greaterefficiency, better decision-making abilities in flexible contexts,better visual acuity, better memory, faster learning rate, bettercognitive control to ignore distractors, faster rate of adaptation,higher kill-death ratio, more damage dealt, less damage accrued, moredamage blocked, less time spent on objective, more kills or less deathsby objective, more critical damage, more healing dealt, better healingaccrued, more assists, and more final blows.

According to another embodiment, a method for sensorimotor training isprovided, comprising: (i) receiving information about at least onepersons' movements when interacting with a virtual environment; (ii)monitoring the movements to assess at least one of the persons'performance interacting with the virtual environment, whereinperformance comprises at least one of speed and accuracy of movement;and (iv) providing feedback to at least one of the persons about theassessment of their performance, wherein the feedback comprises at leastone of visual, auditory, or somatosensory stimulation, and wherein thefeedback is designed to improve the persons' performance.

The feedback further comprises at least one of indicating the speed ofat least one of the persons' movements, and indicating the accuracy ofat least one of the persons' movements. The feedback can also comprisepresenting at least one target, removing at least one of the targetswhen at least one of the players' initiate movements, and re-presentingat least one of the targets after at least one of the players completeat least one movement. The feedback can also comprise explosive feedbackindicating whether or not a target was hit, and where the target washit. The feedback can also comprise a change in the color of a cursor orcrosshairs indicating whether or not a target was hit, and whether ornot a movement was correct or accurate. The feedback can also compriseauditory tones or sounds or somatosensory stimulation presented duringor immediately following a player's movements. The feedback can alsocomprise a visual representation summarizing a player's performance overa period of time.

According to another embodiment, a method for training sensorimotorperformance is provided, comprising: (i) presenting sensory stimulationto at least one person, wherein the sensory stimulation comprises atleast one of visual, auditory, or somatosensory stimulation, and whereinthe sensory stimulation is rendered to correspond to a virtualenvironment; (ii) presenting distracting sensory stimulation to at leastone of the persons, wherein the distracting sensory stimulationcomprises at least one of visual, auditory, or somatosensorystimulation; (iii) receiving information about at least one of thepersons' movements when interacting with the virtual environment; (iv)monitoring the movements to assess at least one of the persons'performance interacting with the virtual environment, whereinperformance comprises at least one of speed and accuracy of movement;and (v) manipulating the distracting sensory stimulation presented to atleast one of the persons based on the assessment of their performance,wherein the manipulation of the distracting sensory stimulation isdesigned to improve the persons' performance.

The manipulation of the distracting sensory stimulation furthercomprises at least one of changing the distance between distractorstimuli and target stimuli, changing the time between presentation ofdistractor stimuli and target stimuli, changing thestimulus-onset-asynchrony between distractor stimuli and target stimuli,changing the presentation duration of at least one distractor stimulus,changing the contrast of at least one distractor stimulus, changing thetransparency of at least one distractor stimulus, changing the size ofat least one distractor stimulus, changing the color of at least onedistractor stimulus, changing the shape of at least one distractorstimulus, changing the texture of at least one distractor stimulus,changing the temporal frequency of at least one distractor stimulus,changing the speed of motion of at least one distractor stimulus,changing the direction of motion of at least one distractor stimulus,changing the relative velocity between at least one distractor stimulusand at least one target stimulus. The distractor stimuli can alsocomprise auditory tones or sounds or somatosensory stimulation.

According to another embodiment, a method for sensorimotor training isprovided, comprising: (i) receiving information about at least onepersons' movements when interacting with a virtual environment; (ii)monitoring the movements to assess at least one of the persons'performance interacting with the virtual environment; and (iii)manipulating the relationship between at least one of the persons'movements and their interaction with the virtual environment, whereinthe manipulation is designed to improve the persons' performance. Theperformance can comprise at least one of speed and accuracy.

The manipulation of the relationship between the persons' movements andtheir interaction with the virtual environment further comprisesmanipulating at least one of position, velocity, acceleration,higher-order temporal derivatives of the position, orientation, angularvelocity, angular acceleration, higher-order temporal derivatives oforientation, joint angle, angular velocity of joint angle, angularacceleration of joint angle, and higher-order temporal derivatives ofjoint angle.

According to another embodiment, a method for sensorimotor assessment isprovided, comprising: (i) presenting sensory stimulation to a pluralityof persons, wherein the sensory stimulation comprises at least one ofvisual, auditory, or somatosensory stimulation, and wherein the sensorystimulation is rendered to correspond to a virtual environment; (ii)receiving information about at least one of the persons' movements wheninteracting with the virtual environment; (iii) monitoring the movementsto assess each person's performance interacting with the virtualenvironment, wherein performance comprises at least one of speed andaccuracy of movement; (iv) applying at least one of pattern recognition,pattern classification, and machine learning operations to analyzeperformance assessment of the plurality of persons; and (v) comparingthe performance assessment of an individual player with that of theplurality of players.

The pattern recognition, pattern classification, and machine learningoperations further comprise at least one of correlation, canonicalcorrelation, sum of squared difference, least-squares, partial leastsquares, nearest neighbor, Mahalanobis distance, regression, multiplelinear regression, logistic regression, polynomial regression, generallinear model, principal components analysis (PCA), singular valuedecomposition (SVD), factor analysis, principal components regression,independent components analysis (ICA), multidimensional scaling,dimensionality reduction, maximum likelihood classifier, maximum aposteriori classifier, Bayesian classifier, Bayesian decision rule,radial basis functions, linear discriminant analysis, regularizeddiscriminant analysis, general linear discriminant analysis, flexiblediscriminant analysis, penalized discriminant analysis, mixturediscriminant analysis, Fischer linear discriminant, regularization,density estimation, naive Bayes classifier, mixture model, Gaussianmixtures, minimum description length, cross-validation, bootstrapmethods, EM algorithm, Markov chain Monte Carlo (MCMC) methods,regression trees, classification trees, boosting, AdaBoost, gradientboosting, neural network classifier, projection pursuit, projectionpursuit regression, support vector machine, support vector classifier,K-means clustering. vector quantization, k-nearest-neighbor classifier,adaptive nearest-neighbor classifier, cluster analysis, clusteringalgorithms, k-medoids, hierarchical clustering, sparse principalcomponents, non-negative matrix factorization, nonlinear dimensionreduction, undirected graph models, statistical learning, supervisedlearning, and unsupervised learning.

According to another embodiment, a method for sensorimotor assessment isprovided, comprising: (i) determining the state of a virtual environmentby reading from computer memory; (ii) receiving information about atleast one persons' movements when interacting with the virtualenvironment; and (iii) monitoring the movements to assess each person'sperformance interacting with the virtual environment, whereinperformance comprises at least one of speed and accuracy of movement.

According to another embodiment, a system for sensorimotor assessment isprovided, comprising: a computer program configured to present sensorystimulation to at least one person. The sensory stimulation comprises atleast one of visual, auditory, or somatosensory stimulation, and isrendered to correspond to a virtual environment. The computer programreceives input from at least one input controller configured to provideinput signals to the computer program that allow at least one of thepersons to interact with the virtual environment. The computer programchanges the state of the virtual environment based on the input signals,and the computer program re-renders the sensory stimulation according tothe changes of state of the virtual environment. The computer program isfurther configured to evaluate the input signals to assess at least oneof the persons' performance interacting with the virtual environment,wherein performance comprises at least one of speed and accuracy ofmovement.

According to another embodiment, a system for sensorimotor training isprovided, comprising: (i) a computer program configured to presentsensory stimulation to at least one person, wherein the sensorystimulation comprises at least one of visual, auditory, or somatosensorystimulation, and wherein the sensory stimulation is rendered tocorrespond to a virtual environment; and (ii) at least one inputcontroller configured to provide input signals to the computer programthat allow at least one of the persons to interact with the virtualenvironment. The computer program is also configured to: (i) change thestate of the virtual environment based on the input signals; (ii)re-render the sensory stimulation according to the changes of state ofthe virtual environment; (iii) evaluate the input signals to assess atleast one of the persons' performance interacting with the virtualenvironment, wherein performance comprises at least one of speed andaccuracy of movement; and (iv) manipulate the sensory stimulationpresented to at least one of the persons based on the assessment oftheir performance, wherein the manipulation of sensory stimulation isdesigned to improve the persons' performance.

According to another embodiment, a system for sensorimotor training isprovided, comprising: (i) a computer program configured to presentsensory stimulation to at least one person, wherein the sensorystimulation comprises at least one of visual, auditory, or somatosensorystimulation, and wherein the sensory stimulation is rendered tocorrespond to a virtual environment; and (ii) at least one inputcontroller configured to provide input signals to the computer programthat allow at least one of the persons to interact with the virtualenvironment, wherein the computer program changes the state of thevirtual environment based on the input signals, and the computer programre-renders the sensory stimulation according to the changes of state ofthe virtual environment. The computer program is also configured to: (i)evaluate the input signals to assess at least one of the persons'performance interacting with the virtual environment, whereinperformance comprises at least one of speed and accuracy of movement;and (ii) provide feedback to at least one of the persons about theassessment of their performance, wherein the feedback comprises at leastone of visual, auditory, or somatosensory stimulation, and wherein thefeedback is designed to improve the persons' performance.

According to another embodiment, a system for training sensorimotorperformance is provided, comprising: (i) a computer program configuredto present sensory stimulation and distracting sensory stimulation to atleast one person, wherein the sensory stimulation comprises at least oneof visual, auditory, or somatosensory stimulation, wherein the sensorystimulation is rendered to correspond to a virtual environment, whereinthe distracting sensory stimulation comprises at least one of visual,auditory, or somatosensory stimulation; and (ii) at least one inputcontroller, configured to provide input signals to the computer programthat allow at least one of the persons to interact with the virtualenvironment. The computer program is configured to: (i) change the stateof the virtual environment based on the input signals; (ii) re-rendersthe sensory stimulation according to the changes of state of the virtualenvironment; (iii) evaluate the input signals to assess at least one ofthe persons' performance interacting with the virtual environment,wherein performance comprises at least one of speed and accuracy ofmovement; and (iv) manipulate the distracting sensory stimulationpresented to at least one of the persons based on the assessment oftheir performance, wherein the manipulation of distracting sensorystimulation is designed to improve the persons' performance.

According to another embodiment, a system for sensorimotor training isprovided, comprising: (i) a computer program configured to presentsensory stimulation to at least one person, wherein the sensorystimulation comprises at least one of visual, auditory, or somatosensorystimulation, and wherein the sensory stimulation is rendered tocorrespond to a virtual environment; and (ii) at least one inputcontroller configured to provide input signals to the computer programthat allow at least one of the persons to interact with the virtualenvironment. The computer program is configured to: (i) changes thestate of the virtual environment based on the input signals, and thecomputer program re-renders the sensory stimulation according to thechanges of state of the virtual environment; (ii) evaluate the inputsignals to assess at least one of the persons' performance interactingwith the virtual environment, wherein performance comprises at least oneof speed and accuracy of movement; and (iii) manipulate the relationshipbetween at least one of the persons' movements and their interactionwith the virtual environment, wherein the manipulation is designed toimprove the persons' performance.

According to another embodiment, a system for sensorimotor assessment isprovided, comprising: (i) a computer program configured to presentsensory stimulation to at least one person, wherein the sensorystimulation comprises at least one of visual, auditory, or somatosensorystimulation, and wherein the sensory stimulation is rendered tocorrespond to a virtual environment; and (ii) at least one inputcontroller, configured to provide input signals to the computer programthat allow at least one of the persons to interact with the virtualenvironment. The computer program is also configured to: (i) change thestate of the virtual environment based on the input signals; (ii)re-render the sensory stimulation according to the changes of state ofthe virtual environment; (iii) evaluate the input signals to assess atleast one of the persons' performance interacting with the virtualenvironment, wherein performance comprises at least one of speed andaccuracy of movement; (iv) apply at least one of pattern recognition,pattern classification, and machine learning operations to analyzeperformance assessment of the plurality of persons; and (v) compare theperformance assessment of an individual player with that of theplurality of players.

According to another embodiment, a system for detecting cheating invideo games and eSports is provided, comprising: (i) a computer program;and (ii) at least one input controller, configured to provide inputsignals to the computer program. The computer program is also configuredto: (i) evaluate the input signals to assess at least one persons'performance interacting with a virtual environment, wherein performancecomprises at least one of speed and accuracy of movement; (ii) apply atleast one of pattern recognition, pattern classification, and machinelearning operations to analyze performance assessment of the pluralityof persons; and (iii) compare the performance assessment of anindividual player with that of the plurality of players.

According to another embodiment, a system for sensorimotor assessment isprovided, comprising: (i) a computer program; and (ii) at least oneinput controller, configured to provide input signals to the computerprogram. The computer program is also configured to: (i) determine thestate of a virtual environment by reading from computer memory; and (ii)evaluate the input signals to assess at least one persons' performanceinteracting with the virtual environment, wherein performance comprisesat least one of speed and accuracy of movement.

According to another embodiment, a system for improving sensorimotorperformance is provided, comprising: (i) a computer program configuredto present sensory stimulation to at least one person, wherein thesensory stimulation comprises at least one of visual, auditory, orsomatosensory stimulation, and wherein the sensory stimulation isrendered to correspond to a virtual environment; and (ii) at least oneinput controller configured to provide input signals to the computerprogram that allow at least one of the persons to interact with thevirtual environment. The computer program is configured to: (i) changethe state of the virtual environment based on the input signals, (ii)re-render the sensory stimulation according to the changes of state ofthe virtual environment; (iii) evaluate the input signals to assess atleast one of the persons' movements interacting with the virtualenvironment; and (iv) manipulate the evaluation of the input signals tocorrect for systematic biases in the persons' movements.

Various embodiments can be implemented, for example, using one or morewell-known computer systems, such as computer system 1500 shown in FIG.15 . Computer system 1500 can be any well-known computer capable ofperforming the functions described herein.

Computer system 1500 includes one or more processors (also calledcentral processing units, or CPUs), such as a processor 1504. Processor1504 is connected to a communication infrastructure or bus 1506.

One or more processors 1504 may each be a graphics processing unit(GPU). In an embodiment, a GPU is a processor that is a specializedelectronic circuit designed to process mathematically intensiveapplications. The GPU may have a parallel structure that is efficientfor parallel processing of large blocks of data, such as mathematicallyintensive data common to computer graphics applications, images, videos,etc.

Computer system 1500 also includes player input/output device(s) 1503,such as monitors, keyboards, pointing devices, etc., that communicatewith communication infrastructure 1506 through player input/outputinterface(s) 1502.

Computer system 1500 also includes a main or primary memory 1508, suchas random access memory (RAM). Main memory 1508 may include one or morelevels of cache. Main memory 1508 has stored therein control logic(i.e., computer software) and/or data.

Computer system 1500 may also include one or more secondary storagedevices or memory 1510. Secondary memory 1510 may include, for example,a hard disk drive 1512 and/or a removable storage device or drive 1514.Removable storage drive 1514 may be a floppy disk drive, a magnetic tapedrive, a compact disk drive, an optical storage device, tape backupdevice, and/or any other storage device/drive.

Removable storage drive 1514 may interact with a removable storage unit1518. Removable storage unit 1518 includes a computer usable or readablestorage device having stored thereon computer software (control logic)and/or data. Removable storage unit 1518 may be a floppy disk, magnetictape, compact disk, DVD, optical storage disk, and/any other computerdata storage device. Removable storage drive 1514 reads from and/orwrites to removable storage unit 1518 in a well-known manner.

According to an exemplary embodiment, secondary memory 1510 may includeother means, instrumentalities or other approaches for allowing computerprograms and/or other instructions and/or data to be accessed bycomputer system 1500. Such means, instrumentalities or other approachesmay include, for example, a removable storage unit 1522 and an interface1520. Examples of the removable storage unit 1522 and the interface 1520may include a program cartridge and cartridge interface (such as thatfound in video game devices), a removable memory chip (such as an EPROMor PROM) and associated socket, a memory stick and USB port, a memorycard and associated memory card slot, and/or any other removable storageunit and associated interface.

Computer system 1500 may further include a communication or networkinterface 1524. Communication interface 1524 enables computer system1500 to communicate and interact with any combination of remote devices,remote networks, remote entities, etc. (individually and collectivelyreferenced by reference number 1528). For example, communicationinterface 1524 may allow computer system 1500 to communicate with remotedevices 1528 over communications path 1526, which may be wired and/orwireless, and which may include any combination of LANs, WANs, theInternet, etc. Control logic and/or data may be transmitted to and fromcomputer system 1500 via communication path 1526.

At least some of the above embodiments are capable of providing some ofthe assessment and training capabilities described in this disclosure.The above embodiments have been tested and used by over 150,000 eSportsplayers. In some embodiments, a system includes a Windows 10 PC with anNVIDIA GTX 1060 graphics card, 21″ computer monitor, and mouse andkeyboard input devices. The software was written in the C# programminglanguage using the Unity game engine. The Unity game engine is across-platform video game engine developed by Unity Technologies that isused for developing video games and simulations for computers, mobiledevices, and gaming consoles. A person of ordinary skill in the artwould recognize that the interface hardware and software could be varieddepending on the type of game skills and games for which training isbeing provided. In some embodiments, a player's performance is assessed,including shooting speed, shooting precision, shooting accuracy,shooting reaction time, along with other assessment metrics as disclosedabove. A person of skill in the art would recognize that a variety ofother gaming skills may be the subject of assessment and training aswell.

In an embodiment, a tangible apparatus or article of manufacturecomprising a tangible computer useable or readable medium having controllogic (software) stored thereon is also referred to herein as a computerprogram product or program storage device. This includes, but is notlimited to, computer system 1500, main memory 1508, secondary memory1510, and removable storage units 1518 and 1522, as well as tangiblearticles of manufacture embodying any combination of the foregoing. Suchcontrol logic, when executed by one or more data processing devices(such as computer system 1500), causes such data processing devices tooperate as described herein.

Based on the teachings contained in this disclosure, it will be apparentto persons skilled in the relevant art(s) how to make and useembodiments using data processing devices, computer systems and/orcomputer architectures other than that shown in FIG. 15 . In particular,embodiments may operate with software, hardware, and/or operating systemimplementations other than those described herein.

It is to be appreciated that the Detailed Description section, and notany other section, is intended to be used to interpret the claims. Othersections can set forth one or more but not all exemplary embodiments ascontemplated by the inventor(s), and thus, are not intended to limitthis disclosure or the appended claims in any way.

While this disclosure describes exemplary embodiments for exemplaryfields and applications, it should be understood that the disclosure isnot limited thereto. Other embodiments and modifications thereto arepossible, and are within the scope and spirit of this disclosure. Forexample, and without limiting the generality of this paragraph,embodiments are not limited to the software, hardware, firmware, and/orentities illustrated in the figures and/or described herein. Further,embodiments (whether or not explicitly described herein) havesignificant utility to fields and applications beyond the examplesdescribed herein.

Embodiments have been described herein with the aid of functionalbuilding blocks illustrating the implementation of specified functionsand relationships thereof. The boundaries of these functional buildingblocks have been arbitrarily defined herein for the convenience of thedescription. Alternate boundaries can be defined as long as thespecified functions and relationships (or equivalents thereof) areappropriately performed. Also, alternative embodiments can performfunctional blocks, steps, operations, methods, etc. using orderingsdifferent than those described herein.

References herein to “one embodiment,” “an embodiment,” “an exampleembodiment,” or similar phrases, indicate that the embodiment describedcan include a particular feature, structure, or characteristic, butevery embodiment can not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it would be within the knowledge of persons skilled in therelevant art(s) to incorporate such feature, structure, orcharacteristic into other embodiments whether or not explicitlymentioned or described herein. Additionally, some embodiments can bedescribed using the expression “coupled” and “connected” along withtheir derivatives. These terms are not necessarily intended as synonymsfor each other. For example, some embodiments can be described using theterms “connected” and/or “coupled” to indicate that two or more elementsare in direct physical or electrical contact with each other. The term“coupled,” however, can also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other.

The breadth and scope of this disclosure should not be limited by any ofthe above-described exemplary embodiments, but should be defined only inaccordance with the following claims and their equivalents.

What is claimed is:
 1. A computer-implemented method for improving performance of a player of a game, the method comprising: receiving, by at least one processor, an input from the player of the game; assessing, by the at least one processor, based on the input from the player, a performance of the player; and providing feedback to the player to improve the player's performance in the game, based on the assessing.
 2. The method of claim 1, wherein the assessing is a statistical analysis.
 3. The method of claim 1, wherein the feedback comprises at least one of: audio feedback, visual feedback, and somatosensory stimulation; and wherein the feedback is provided with each shot or movement.
 4. The method of claim 1, wherein the feedback comprises removing a target when the player initiates a movement and then re-presenting the target after a movement or after a shot.
 5. The method of claim 1, wherein the feedback comprises one of a score over time and performance over time.
 6. The method of claim 1, wherein the feedback comprises a score in comparison to other players.
 7. The method of claim 1, wherein the assessing comprises analyzing a movement trajectory based on the input.
 8. The method of claim 1, wherein the assessing comprises analyzing one or more of speed, precision, accuracy, and reaction time of the input.
 9. The method of claim 1, wherein the input is received from a device selected from the group consisting of: a mouse; a joystick; an accelerometer; a pointing device; a motion capture device; a Wii remote controller; an eye tracker; and a computer vision system.
 10. A device for improving performance of a player of a game, the device comprising: a memory; and a processor to: receive an input from the player of the game; assess, based on the input from the player, a performance of the player; and provide feedback to the player to improve the player's performance in the game, based on the assessing.
 11. The device of claim 10, wherein the assessing is a statistical analysis.
 12. The device of claim 10, wherein the feedback comprises at least one of: audio feedback, visual feedback, and somatosensory stimulation; and wherein the feedback is provided with each shot or movement.
 13. The device of claim 10, wherein the feedback comprises removing a target when the player initiates a movement and then re-presenting the target after a movement or after a shot.
 14. The device of claim 10, wherein the feedback comprises one of a score over time and performance over time.
 15. The device of claim 10, wherein the feedback comprises a score in comparison to other players.
 16. The device of claim 10, wherein the assessing comprises analyzing a movement trajectory based on the input.
 17. The device of claim 10, wherein the assessing comprises analyzing one or more of speed, precision, accuracy, and reaction time of the input.
 18. The device of claim 10, wherein the input is received from a device selected from the group consisting of: a mouse; a joystick; an accelerometer; a pointing device; a motion capture device; a Wii remote controller; an eye tracker; and a computer vision system.
 19. A computer-implemented method for improving performance of a player of a game, the method comprising: receiving, by at least one processor, an input via a device, the input controlling a movement; assessing, by the at least one processor, based on the movement, a metric; and providing feedback to a player to improve the performance of the player in the game, based on the assessing.
 20. The method of claim 19, wherein the device is selected from the group consisting of: a mouse; a joystick; an accelerometer; a pointing device; a motion capture device; a Wii remote controller; an eye tracker; and a computer vision system. 